Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Dockerfile configuration #195

Open
wants to merge 32 commits into
base: develop
Choose a base branch
from

Conversation

mcdonnnj
Copy link
Member

@mcdonnnj mcdonnnj commented Mar 6, 2024

🗣 Description

This pull request represents a rework of this project's Docker configuration. It includes the following pull requests:

💭 Motivation and context

We have been striving to use configurations that are better at producing repeatable builds in downstream projects. This represents a backporting of a lot of that work to this skeleton so that all of our Docker projects are similarly configured.

🧪 Testing

Automated tests pass.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All relevant repo and/or project documentation has been updated
    to reflect the changes in this PR.
  • Tests have been added and/or modified to cover the changes in this PR.
  • All new and existing tests pass.

✅ Post-merge checklist

  • Create a release.

mcdonnnj and others added 30 commits February 19, 2024 03:12
This helps ensure that when a Docker image is built the expacted source
image is used regardless of what repository is configured as the
default on the host system. It also makes our Dockerfiles more
seamlessly convertible to using the GitHub Container Registry or any
other Open Container Initiative (OCI) compatible registry.
Use the full tag that includes the Alpine Linux version to ensure the
pulled image is always the same.
Use the full path for source container images
Instead of downloading the source archive, extracting it, and then
installing it with pip we instead just let pip directly install the
package.
Since we are now installing cisagov/skeleton-python-library directly
with pip we no longer need these OS packages.
Now that we are not overwriting the internal Python package file the
text we look for must match what is output by default. The Docker
Compose secret configuration is left in place to continue to serve as
an example and to be leveraged for a future update to
cisagov/skeleton-python-library that can provide similar functionality
to what was removed in this project.
…hon-library_directly

Install cisagov/skeleton-python-library directly with `pip`
We should not blindly upgrade all pre-installed packages. This can
create inconsistent build results due to changes in installed versions.
Pin the versions of the pip, setuptools, and wheel packages that are
installed.
We can move this instruction to the end of the Dockerfile now that we
are no longer working with files in the Docker container when building.
…eatable

Pin Python package versions and improve build repeatability
Instead of relying on `pip3` being on the PATH we instead call the
module through the Python executable. This ensures that the `pip` being
used is in the same environment as the `python3` being used.
Using a virtual environment is a Python best practice. We also
consolidate all of the Python dependency installation steps into a
single RUN instruction. This ensures that Python setup is cached in one
layer and mirrors the logical organization of this being a single step.
Since we cannot use long options on Alpine Linux we should explain what
the short options we are using do. I also changed the order of options
so that they are in alphabetical order.

Co-authored-by: Shane Frasier <[email protected]>
This configuration includes a Pipfile configuration file and the
generated Pipfile.lock file that pins to specific versions for the
Python dependencies for this project. This will help us ensure
repeatable builds. The pipenv package is added as a developmental
requirement to support these files.
Now that we have a pipenv configuration we will use it to install the
Python dependencies for the image. The `build` workflow is updated to
no longer pass the VERSION build argument in line with this change.
Switch to using a multi-stage build in the Dockerfile. This reduces
image size since pipenv and its dependencices are not needed in the
final image. It also ensures that the system Python environment is
unmodified.
Install the core Python packages (pip, setuptools, and wheel) into the
system Python environment before installing pipenv. This keeps things
consistent with our usual approach to Python environments.
The comment references a command that is no longer being run.

Co-authored-by: Shane Frasier <[email protected]>
…tion

Install Python dependencies with `pipenv`
Change the tags used in the table to match the version of the project.
Previously "1.2.3" was used as an example version but there is no
reason not to use the real version of the image.
Update the Dockerfile and testing to accommodate changes in the new
version.
@mcdonnnj mcdonnnj added breaking change This issue or pull request involves changes to existing functionality improvement This issue or pull request will add or improve functionality, maintainability, or ease of use version bump This issue or pull request increments the version number labels Mar 6, 2024
@mcdonnnj mcdonnnj self-assigned this Mar 6, 2024
@mcdonnnj mcdonnnj requested a review from a team March 6, 2024 20:32
Dockerfile Outdated Show resolved Hide resolved
Dockerfile Outdated Show resolved Hide resolved
Copy link
Member

@dav3r dav3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 after symlinkgate has been resolved.

Copy link
Member

@felddy felddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fully charged! 🔋
You deserve an award for this modernization! 🏆
I had a few comments.

# because the ln binary in Alpine Linux does not support long flags. The -f instructs
# ln to remove the existing file and the -s instructs ln to create a symbolic link.
###
COPY --from=compile-stage --chown=${CISA_USER}:${CISA_GROUP} ${VIRTUAL_ENV} ${VIRTUAL_ENV}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL you can chown during a COPY in a Dockerfile. Bravo.

# cache. This results in a smaller final image, at the cost of
# slightly longer install times.
###
RUN python3 -m pip install --no-cache-dir --upgrade \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is the compile-stage of the build, do we need to optimize for layer count. i.e., does it still make sense to concatenate each command with && versus having discreet RUNs.

I ask because I think you will get a more localized error statement when something fails when the command is decomposed. There may be some other gains to be had as well with caching, and parallelization.

&& pip3 install --no-cache-dir --requirement requirements.txt \
&& ln -snf /run/secrets/quote.txt src/example/data/secret.txt \
&& rm sourcecode.tgz
# Note that we symlink the Python binary in the venv to the system-wide Python so that
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this comment be moved down to after the COPY but before the RUN ln ... line?

cisagovbot pushed a commit that referenced this pull request Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change This issue or pull request involves changes to existing functionality improvement This issue or pull request will add or improve functionality, maintainability, or ease of use version bump This issue or pull request increments the version number
Projects
Status: Reviewer approved
Development

Successfully merging this pull request may close these issues.

4 participants